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I.  SUMMARY 


The  research  project  was  designed  to  delineate  principles  that  underlie  the  perception 
of  complex  auditory  patterns.  During  the  granting  period,  nine  lines  of  research  were 
conducted  that  investigated  various  aspects  of  complex  auditory  perception.  These  research 
efforts  largely  focussed  on  perception  of  speech  sounds,  and  provided  important  information 
about  three  aspects  of  perception.  Several  of  the  projects  clarified  the  role  that  the  listener’s 
knowledge  of  English  words  can  play  in  decoding  speech.  Additional  studies  examined  how 
lower-level  representations  (spectral  patterns,  high-frequency  sublexical  patterns)  are 
processed.  Across  a  number  of  the  research  efforts,  attentional  effects  were  investigated,  to 
determine  how  they  modulate  other  processing.  Collectively,  the  research  effort  made 
significant  progress  in  clarifying  how  human  listeners  decode  very  complex  sounds. 
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n.  RESEARCH  OBJECTIVES 


The  objective  of  the  research  project  is  to  delineate  principles  that  underlie  the 
perception  of  conrplex  auditory  patterns.  The  stimuli  used  are  speech  and  musical  patterns  of 
varying  complexity.  A  wide  array  of  experimental  procedures  and  analyses  are  used  to  try  to 
determine  properties  that  are  true  of  the  perception  of  complex  auditory  patterns  across 
stimulus  domains.  In  addition,  we  also  are  interested  in  discovering  any  principles  that  are 
domain  specific  (e.g.,  as  "categorical  perception"  has  traditionally  been  claimed  to  be  a 
principle  of  perception  specific  to  the  speech  domain).  The  various  experimental 
investigations  in  the  project  may  be  broadly  grouped  into  studies  of  signal-based  factors,  and 
studies  of  listener-based  factors.  The  former  group  includes  experiments  that  explore  how 
properties  of  the  input  signal  determine  perception,  while  the  latter  group  includes  studies  of 
how  listeners’  expectations  influence  perception/performance.  The  former  group  primarily 
focusses  on  early  representations  of  the  signal,  and  the  latter  includes  higher-level  factors 
(including,  but  not  limited  to,  attentional  influences).  The  long-term  goal  of  the  research  is 
to  understand  both  signal-based  and  listener-based  factors,  and  their  interaction  in  the 
perception  of  complex  auditory  patterns. 
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III.  RESEARCH  ACCOMPLISHED 


During  the  period  of  AFOSR  grant  91-0378,  we  conducted  a  large  number  of  studies 
of  the  perception  of  complex  sounds,  working  within  an  information  processing  framework. 
A  fundamental  premise  of  the  information  processing  perspective  is  that  perceptual  and 
cognitive  functioning  may  usefully  be  decomposed  into  levels  of  analysis.  For  understanding 
how  complex  sounds  are  perceived,  this  perspective  entails  providing  a  specification  of  what 
each  level  of  analysis  is,  and  the  relationship  of  each  level  to  other  levels.  In  specifying  a 
level  of  analysis,  there  are  many  kinds  of  information  that  we  should  want  to  know.  For 
example,  it  is  important  to  delineate  the  domain  of  operation;  Does  a  process  at  a  particular 
level  of  analysis  only  operate,  say,  on  auditory  stimuli,  or  perhaps  only  on  auditory  signals 
from  one  ear,  or  only  on  signals  with  certain  properties  (e.g.,  musical  sounds),  etc.  To  the 
extent  that  we  can  specify  the  stimulus  properties  that  are  critical  to  an  analysis  at  a  given 
level,  we  understand  the  nature  of  the  system.  Moreover,  we  must  understand  the 
mechanisms  of  processors  at  a  given  level.  Do  they,  for  example,  change  their  output  as  a 
function  of  the  stimulation,  or  are  they  stable  over  time?  Does  the  activation  of  a  particular 
representation  have  any  effect  on  other  representations  at  the  same  level,  or  is  each  one 
indq)endent?  Similarly,  we  should  try  to  understand  how  the  activity  at  one  level  of  the 
system  affects  the  behavior  of  processors  at  other  levels. 
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Broadly  speaking,  the  research  conducted  in  our  laboratory  can  be  characterized  as 


focussing  on  one  or  more  of  three  topics:  (1)  Lexical  issues  -  what  influence  do  pre¬ 
existing  representations  of  known  words  have  on  the  perception  of  speech?  (2)  Sublexical 
representations  —  what  levels  of  representation  does  the  perceptual  system  employ  between 
the  initial  post-cochlear  signal,  and  the  ultimate  lexical  coding?  (3)  Attentional  allocation  - 
what  are  the  control  processes  guiding  perception  of  complex  sounds,  and  how  do  they  affect 
processing? 

During  the  grant  period,  we  have  made  significant  progress  in  each  of  these  areas. 
Most  of  this  progress  has  been  reviewed  in  our  previous  Annual  Technical  reports. 

Therefore,  we  will  provide  only  brief  summaries  of  the  nine  projects  we  have  undertaken. 
For  relatively  complete  projects,  we  have  already  submitted  copies  of  the  relevant 
manuscripts;  we  will  provide  such  manuscripts  in  the  coming  months  for  the  several  projects 
that  are  approaching  completion. 


A.  An  Empirical  and  Meta-Analvtical  Study  of  Hionemic  Identification 

Recent  studies  using  Ganong’s  (1980)  identification  task  have  produced  discrepant 
results.  This  project  helped  to  resolve  these  discrepancies  by  examining  the  influence  of 
methodological  factors  on  phoneme  identification  and  differences  in  data  analysis  techniques. 
Three  factors  were  examined  across  two  experiments:  position  of  target  phoneme,  phonetic 
contrast,  and  two  task  conditions  in  which  stimulus  quality  (S/N  ratio)  or  cognitive  load 
varied.  A  meta-analysis  was  then  performed  on  the  results  from  all  identification  studies  in 
the  literature,  in  an  effort  to  obtain  additional  insight  into  factors  that  influence  the  task.  The 
experiments  and  meta-analysis  identified  the  importance  of  a  number  of  methodological 
factors  in  affecting  identification,  most  notably,  position  of  the  target  phoneme  (early  or  late 
in  a  word). 
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B.  Lexical  Influences  in  Perceptually  Restoring  Deleted  Phonemes 


A  critical  issue  in  modelling  speech  perception  is  whether  high-level  representations 
(e.g.,  lexical)  can  affect  the  processing  at  lower  levels  (e.g.,  phonemic).  Previous  research, 
using  studies  of  phonemic  restoration,  has  provided  support  for  such  top-down  influences. 
However,  there  have  been  a  number  of  failures  to  find  these  effects  in  restoration  studies. 
This  project  examined  the  coming  and  going  of  lexical  effects,  and  provided  an  account  of 
when  such  effects  will  and  will  not  appear.  Part  I  demonstrated  the  fragility  of  lexical 
effects  and  shows  that  the  originally-reported  lexical  effects  are  in  fact  reliable.  In  Part  II,  a 
new  methodology  was  introduced,  a  methodology  that  provides  very  good  approximations  to 
the  underlying  distributions  of  perceived  intactness  that  are  assumed  in  signal  detection 
analyses  of  phonemic  restoration.  This  methodology  provides  a  much  more  sensitive 
technique  for  determining  the  conditions  that  are  necessary  for  lexical  feedback  to  occur. 
These  conditions  were  created  in  Part  III,  and  a  reliable  lexical  influence  on  phonemic 
perception  resulted.  Collectively,  the  four  experiments  illustrate  that  lexical  activation  does 
influence  lower-level  processing,  and  that  these  influences  are  quite  fragile.  These  results  for 
phonemic  restoration  are  consistent  with  results  from  several  other  domains.  The  theoretical 
implications  of  real  but  fragile  lexical  effects  are  discussed. 


C.  A  Test  for  Lexical  Inhibition 

In  the  very  influential  interactive  model  TRACE,  a  critical  mechanism  is  lexical 
inhibition:  The  activation  of  one  word  is  hypothesized  to  inhibit  the  activation  of  others,  due 
to  competition  between  similar  lexical  items.  This  project,  in  its  preliminary  stages,  provides 
an  empirical  test  of  lexical  inhibition,  with  encouraging  results.  Subjects  were  presented 
with  a  (word  or  nonword)  target  visually,  followed  by  the  dichotic  presentation  of 
monosyllabic  auditory  stimuli.  There  were  three  dichotic  cases:  word-word,  word-nonword, 
and  nonword-nonword.  The  central  question  was  whether  recognizing  a  target  would  be 
hindered  by  the  simultaneous  presentation  of  a  (theoretically  lexically-inhibitory)  word.  In 
fact,  exactly  this  result  was  observed:  words  were  detected  significantly  more  slowly  when 
they  were  competing  with  another  real  word  than  with  a  nonword.  This  initial  result 
provides  preliminary  support  for  TRACE’S  assumptions  regarding  lexical  inhibition. 


D.  The  Migration  Paradigm  and  Lexicalitv 

Models  of  speech  recognition  differ  with  regard  to  the  potential  influence  of  lexical 
and  semantic  knowledge  on  speech  perception.  This  project  addresses  the  issue  of  lexical 
influence  through  a  paradigm  that  has  been  little  used  in  speech  studies,  particularly  in 
English,  namely  the  paradigm  oi  linguistic  unit  migration.  This  technique  entails  presenting 
an  auditory  target  (e.g.,  "preferable")  followed  by  a  dichotic  pair  of  items  where  the 
information  necessary  for  perception  of  the  target  has  been  distributed  between  the  ears  (e.g., 
"priferable"-"glgzanukef').  Migration  of  the  missing  unit  (/E/,  in  this  example),  that  leads  to 
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the  illusory  perception  of  the  target  ("preferable"),  has  been  shown  to  tap  early  stages  during 
the  recognition  process.  This  study  tested  whether  the  migration  phenomenon  is  affected  by 
the  lexicality  of  the  target  and  the  position  of  the  critical  unit  in  the  items.  The  results, 
although  not  yet  definitive  as  to  the  functional  architecture  of  the  speech  perception  system, 
provide  valuable  information  about  the  migration  paradigm  and  its  sensitivity  to  several 
experimental  factors. 


E.  Lexical  and  Sublexical  Facilitation  of  Perception 

Currently,  there  are  two  qualitatively  different  model  classes  in  the  field  of  spoken 
language  understanding.  Autonomous  models  allow  only  bottom-up  information  flow, 
whereas  interactive  models  allow  higher  level  representations  (e.g.,  lexical)  to  affect 
processing  at  lower  levels  (e.g.,  phonemic).  Part  1  of  this  project  included  a  test  of  a 
prediction  that  differentiates  the  two  model  classes:  Is  phoneme  monitoring  faster  for  targets 
in  real  words  than  in  pseudowords,  even  before  the  word  could  in  principle  be  recognized? 
The  results  indicate  that  this  lexical  advantage  does  occur,  in  accord  with  the  predictions  of 
interactive  models.  In  Part  2,  speech  compression  and  expansion  were  used  to  assess  the 
sufficiency  or  necessity  of  bottom-up  evidence  and  of  processing  time  in  accomplishing 
lexical  access.  The  results  of  Parts  1  and  2  suggested  that  in  addition  to  the  lexical  effects 
posited  by  current  models,  sublexical  activation  may  also  plan  an  important  role.  The  two 
experiments  in  Part  3  supported  this  interpretation.  Collectively,  the  experiments  in  the 
current  study  support  interactive  models  of  lexica!  processing,  and  indicate  that  additional 
sublexical  processes  are  necessary  as  well. 


F.  Lexical  Inhibition  and  Attentional  Effects  in  Word  Recognition 

An  interesting  and  relatively  unique  prediction  of  the  interactive  TRACE  model  of 
word  recognition  is  that  the  activation  of  one  lexical  candidate  inhibits  alternative  candidates. 
In  one  of  the  only  direct  tests  of  this  hypothesis,  Frauenfelder,  Segui,  and  Dijkstra  (1990) 
found  no  evidence  for  it.  Three  experiments  in  our  lab  have  shown  that  such  effects  may 
indeed  be  present,  but  difficult  to  find  due  to  attentional  effects  that  are  caused  by  aspects  of 
the  experimental  procedures.  Experiment  1  provided  a  very  clear  replication  of  Frauenfelder 
et  al.’s  null  effect,  using  comparable  conditions  (but  English,  rather  than  French,  stimuli  and 
subjects).  In  Experiment  2,  changes  were  made  in  the  details  of  the  experimental  design, 
involving  the  probabilities  of  words,  nonwords,  and  the  locations  of  phoneme  targets  that 
subjects  were  supposed  to  listen  for.  The  manipulations  succeeded  in  shifting  the  pattern  of 
results,  indicating  that  attentional  allocation  is  sensitive  to  these  factors.  In  a  third 
experiment,  subjects  were  presented  with  speech  in  one  ear,  and  tones  in  the  other,  and  had 
to  monitor  for  both  specified  phonemes  and  frequency  modulations  of  the  tones.  This  dual¬ 
task  methodology  indicated  that  both  attentional  allocation  effects  and  lexical  inhibition 
appear  to  play  a  role  in  the  observed  results. 
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G.  Are  There  Lexical  or  Cognitive  Influences  on  Speech  Adaptation? 

The  selective  adaptation  technique  has  been  used  in  scores  of  studies,  and  has  proven 
very  useful  in  determining  the  types  of  analysis  used  in  perceiving  complex  sounds.  In  this 
project,  the  technique  is  used  in  a  slightly  different  manner  than  usual.  Here,  the  goal  is  to 
find  the  upper  bound  on  adaptation  effects.  Previous  research  has  demonstrated  adaptation- 
induced  effects  at  three  levels  of  processing.  The  most  abstract  of  these  three  is  at  roughly  a 
phonemic  level  of  analysis.  In  four  experiments,  the  current  study  examines  higher  levels  of 
representation.  The  results  indicate  that  adaptation  does  not  operate  at  these  higher  levels. 
For  example,  the  lexical  status  of  the  adaptor  (word  vs.  nonword)  does  not  influence  the  size 
of  the  adaptation  effect.  Similarly,  it  does  not  matter  if  the  test  items  are  words  or  not. 
Moreover,  adaptation  is  unaffected  by  whether  or  not  the  listener  is  attending  to  the  adapting 
sound.  Collectively,  these  experiments  indicate  that  adaptation  does  not  operate  at  higher 
levels  in  the  perception  of  complex  sounds. 


H.  Using  Selective  Adaptation  to  Clarify  Levels  of  Analysis 

A  fundamental  goal  of  an  information  processing  approach  to  speech  perception  is  to 
specify  the  levels  of  analysis  that  occur  between  the  initial  sensory  coding  of  the  signal,  and 
the  recognition  of  the  phonetic  sequence  that  it  conveys.  A  series  of  four  experiments 
provided  evidence  to  support  the  hypothesis  that  there  are  at  least  three  qualitatively  different 
levels  of  analysis  involved  in  the  perception  of  speech  and  other  complex  sounds.  The 
experiments  specify  several  properties  for  the  representations  of  each  level,  including  a  locus 
(peripheral/monaurally-driven  versus  central/binaurally-driven),  a  stimulus  domain,  and  the 
mechanisms  involved  in  response  adjustment  as  a  function  of  repeated  stimulation.  The 
stimulus  domains  for  the  three  levels  are,  respectively,  (1)  processes  that  deal  with  simple 
acoustic  patterns,  (2)  ones  that  integrate  more  complex  acoustic  patterns,  and  (3)  processes 
that  represent  categorical  or  phonetic  information.  The  convergence  among  several  different 
approaches  used  to  determine  levels  of  analysis  supports  the  three-level  model. 


1.  Perceptual  Degradation;  Sound-source  and  Processing  Cycle  Effects 

It  has  been  known  for  four  decades  that  alternating  a  message  between  the  two  ears 
(over  headphones)  causes  a  large  drop  in  intelligibility  when  the  alternation  rate  is 
approximately  3-4  Hz.  At  faster  or  slower  alternation  rates,  intelligibility  is  not  impaired. 

In  addition,  the  critical  alternation  rate  is  not  a  constant  -  it  covaries  linearly  with 
presentation  rate  of  the  message.  This  project  includes  a  large  set  of  experiments  intended  to 
determine  the  mechanisms  that  cause  this  non-monotonic  performance.  The  working 
hypothesis  is  that  the  phenomenon  is  caused  by  the  problematic  interaction  of  two  processes 
that  normally  allow  listeners  to  perceive  complex  sounds  accurately.  One  process  is  sound- 
source  assignment:  Incoming  sounds  must  be  sorted  on  the  basis  of  their  sources  (e.g.,  a 
voice  must  be  disentangled  from  background  sounds).  The  second  process  is  one  which 
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involves  successive  analyses  of  an  individual  sound  stream  (e.g.,  a  series  of  analyses  of  high- 
information  points  within  the  speech  stream).  The  experiments  done  to  date  have 
manipulated  the  fundamental  frequency,  and  the  spectral  makeup,  of  speech  signals,  as  well 
as  more  traditional  tests  using  variation  in  location  (right  versus  left  ear).  At  this  point,  the 
results  have  provided  some  support  for  the  working  hypothesis,  but  further  tests  are  needed, 
and  are  underway. 
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VI.  LIST  OF  INTERACTIONS 


The  P.I.  has  been  extremely  active  in  national  service,  serving  on  the  Editorial 
Boards  of  leading  journals.  During  the  granting  period,  these  Journals  included  Cognition. 
Memory  and  Cognition,  the  Journal  of  Experimental  Psychology:  Human  Perception  and 
Performance,  and  Perception  &  Psychophysics.  Beginning  this  January,  he  will  also  be  on 
the  Editorial  Board  of  the  Journal  of  Memory  and  Language.  He  is  also  a  member  of  the 
Perception  and  Cognition  Review  Panel  for  NIMH.  These  editorial  and  grant  review 
activities  provide  a  rich  source  of  interaction  with  top  scientists  from  around  the  country. 


Presentations  During  the  Grant  Period 

Samuel,  A.  G.  Perceptual  restoration,  perceptual  bias,  priming,  and  pseudowords:  Insights 
from  a  newer  methodology.  Presented  at  the  Psychonomic  Society,  San  Francisco, 
November  1991. 

Pitt,  M,  A.,  and  Samuel,  A.  G.  Is  auditory  word  recognition  serial  or  interactive? 
Presented  at  the  Psychonomic  Society,  San  Francisco,  November  1991. 

Samuel,  A.  G.  Probing  words  and  pseudowords:  Evidence  for  interactive  activation  models 
of  word  recognition.  Presented  at  the  Max  Planck  Institute  for  Psycholinguistics, 
Nijmegen  (the  Netherlands),  August  1992. 
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Samuel,  A.  G.  Levels  of  analysis  and  the  roles  of  attention  in  the  perception  of  speech  and 
music.  Presented  at  the  New  York  Academy  of  Sciences,  May  1993. 

Wurm,  L.  H.,  and  Samuel,  A.  G.  Can  lexical  knowledge  inhibit  phoneme  perception? 
Presented  at  the  Acoustical  Society  of  America,  Denver,  October  1993. 

Samuel,  A.  G.  Is  perceptual  degradation  caused  by  a  confused  sound  source  assignment 
process?  Presented  at  the  Psychonomic  Society,  Washington,  D.  C.,  November 
1993. 

Samuel,  A.  G.  What  makes  our  perceptual  system  fail?  Presented  at  the  Psychology 
Department,  Princeton  University,  December  1993. 

Samuel,  A.  G.  Levels  of  analysis  of  speech  and  other  complex  sounds.  Presented  at  the 
Laboratory  of  Experimental  Psychology,  Brussels  (Belgium),  April  1994. 

Samuel,  A.  G.  Lexical  and  sublexical  feedback  in  auditory  word  recognition.  Presented  at 
the  Workshop  on  Word  Recognition  in  the  Spoken  and  Visual  Modality,  Brussels 
(Belgium),  April  1994. 

Wurm,  L.  H.,  and  Samuel,  A.  G.  Attentional  allocation  during  speech  perception:  Evidence 
from  phoneme  monitoring.  Presented  at  the  Acoustical  Society  of  America, 
Cambridge  (Mass.),  June  1994. 
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